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For an arbitrary complex matrix A we consider (1) the set of all matrices B such that ABA = A 
and AB is Hermitian and (2) the set of all matrices B such that ABA = A and BA is Hermitian. It is shown 
that if B is in (1) then x = By is a least-squares solution oiAx—y and that if B is in (2) then x = By is 
the solution of minimum Euclidian norm of the consistent systems Ax = y. The connection is exposed 
between the properties of the generalized inverses in (a) and (b) and the fact that among all matrices 
X satisfying AXA — A, that with minimum Euclidian norm is the Moore-Penrose inverse of A. 
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1 . Introduction 

In this paper we discuss some properties of certain classes of generalized inverses, give some 
new results and some enlightening connections between some known results. 

It is shown how to construct (a) the set of all matrices B such that ABA = A and AB is Her- 
mitian and (b) the set of all matrices B such that ABA = A and BA is Hermitian. The generalized 
inverses in (a) include the normalized generalized inverse of Rohde [IS] 1 and have been em- 
ployed by Decell and Odell [3] in constructing the fixed point probability vector of regular or 
ergodic transition matrices. The generalized inverses in (b) include the Goldman-Zelen weak 
generalized inverse [4] and these authors have utilized a property of their inverse which we men- 
tion in the next paragraph. 

It is well known [12] that given Ax = y, with A and y a given matrix and vector respectively, 
xo = A + y, where A + is the Moore-Penrose generalized inverse, is the least-squares solution of 
minimum Euclidian norm; in particular, the solution of minimum norm when the system is con- 
sistent. In what follows, we show that if B is in (b), then xo = By is the solution of minimum Euclidian 
norm when the system is consistent. It is further shown that if B is in (a), then x = By is a least- 
squares solution of the system. We then show that the only matrix B such that xo = By is the least- 
squares solution of minimum norm is B = A + . This is known [12]. The proof here shows the con- 
nection between this property and the weaker classes of generalized inverses mentioned above. 
All of the proofs assign a definitive role to a well-known property of perpendicular projections. 
The connection is shown between the properties of the generalized inverses in (a) and (b) and the 
fact that among all matrices B such that ABA =A, that with minimum Euclidian norm is A + . 

2. Definitions and Preliminaries 

We consider matrices with complex entries. For any matrix M we denote by p(M), R{M ), N(M) 
and M* the rank, range, nullspace and conjugate transpose respectively of M. By the norm, ||/V/||, 
of any matrix M, we mean the Euclidian norm: ||/W|| 2= trace (M*M)= trace {MM*). For any column 
vector*, ||*|| 2 = **:£, and we observe that \\M\\ 2 is the sum of the squared norms of the column vectors 
of/W. 
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We use the terminology for generalized inverses previously employed [7, 8, 9]. For a given 
matrix A, C\(A) is the set of all matrices B such that ABA = A; C 2 (A) is the set of all matrices B such 
that BeC\(A) and BAB = B; C :i (A) is the set of all matrices BeC 2 (A) such that AB is Hermitian; C : y(A) 
is the set of all matrices BeC 2 (A) such that BA is Hermitian; C 4 (A) is the set of all matrices B such 
that BeC 3 (A) and BeC 3 >(A). We call a matrix BeC^A) a C-inverse of A, £=1,2, 3, 3', 4. We note that 
C^A), the intersection of C^(A) and C 3 '(/4), is a single matrix, the unique Moore-Penrose generalized 
inverse [11]. We observe that if BeCj(A) then AB amdBA are projections and that p(AB)= p(BA)=p(A) 
[15]. In particular, if i is 3, 3' or 4, then ^5, ZL4 or both, respectively, are perpendicular projections. 
We reserve the notation A + for the C\ -inverse of any matrix A. 

We consider the system of equations 

Ax = y, (1) 

where A and y are given. If yeR(A), i.e., the system is consistent, then it is known [6] that there 
exists a solution xoeR(A*), which is uniquely determined and is the solution of minimum norm: 
IMI ^ IHI? where x is any solution, with equality only if x = Xq. The proof is rather simpler than that 
[6] cited. Any solution x can be written x=u J rv, with ueR(A*) and veN(A). Then clearly u is a 
solution. Let x\ = ii\ J rv\ and xy — w>-\-v 2 be any two solutions with iiieR{A*), v\eN(A), £=1, 2. 
Then x\ — x 2 = {u\ — u 2 ) + (^i — v 2 ) is in N(A), which implies that {x\ — x 2 ) — (v\ —V2) — u\ ~ u 2 
is in N(A). But (u\ — u 2 )eR(A*) and hence ii] = u 2 . Thus any solution can be written x = Xo + v, 
with xoeR(A*) uniquely determined and veN(A). Clearly ||^|| 2 — IM| 2 + ||f|| 2 ^ IM| 2 - In what follows, 
we therefore refer to the solution of minimum norm of a consistent system. 
It is well known that if (1) is inconsistent, then any solution of 

A*Ax = A*y (2) 

is a least-squares solution of (1). For, let x be any vector, x, satisfy (2), d = Ax-y and d 1 =Ax l -y. 
Then, by (2), d*A = and since d x ~d = A(x,-x) we have rf*(d, -d) =0. Thus d*d\ = d?d= d*d x . 
From the Cauchy inequality, {d*d) {d*d x ) ^ (dfd)*, we then have d*d ^ dfdi , with equality if and 
only if d= d x . If equality is the case, then d*A = and x is also a least-squares solution. Conversely, 
if x andxi are least-squares solutions then d= d u for then (x — x\)eN(A*A)=N(A) which implies 
Ax = Ax\. Thus every solution of (2) gives the same vector of deviations d = Ax — y, and, of necessity, 
the same minimum sum of squares. By what has been said in the last paragraph above, there exists 
a unique least-squares solution of minimum norm. Penrose [12] calls this least-squares solution the 
"best approximate solution" of (1), but he applies this concept and definition to more general sys- 
tems than (1). 

If (1) is consistent, any solution is a least-squares solution (zero sum of squares) and the 
solution of minimum norm is the least-squares solution of minimum norm. Thus the Penrose 
definition of "best approximate solution" is meaningful, and always exists for the system (1), 
without reference to the consistency of (1). 

Lemma 1 gives a well-known property of perpendicular projections and Lemma 2 is an easy 
consequence of Lemma 1. We include both for ready reference and clarity. 

Lemma 1. Let E be a projection. Then ||x|| ^ ||Ex||, for every x, if and only if E=E*. In that 
case, ||x|| = ||Ex|| if and only if xeR(E). 

PROOF. The first statement is a known theorem [5, p. 147]. To see the second statement, 
let x = u + v with ueR(E) and veN(E). Then \\Ex\\ 2 = x*Ex= u*u. But ||%|| 2 = ***= u*u if and only 
if xeR(E). 

Lemma 2. Let X be any matrix and E and F be Hermitian projections. If Xo= EXF, we have 
||Xo|| ^ ||X||, with equality if and only ifX = X . 

PROOF. Define Y=EX. Then by Lemma 1, ||y/|| ^ ||x,-|| where y and x% are the ith column of 
Y and X respectively. Thus we have ||F|| ^ \\X\\ with equality if and only if Y = X. Now X$ = FY*, 
and by the same argument \\X \\ ^ ||F||, with equality if and only if X = Y. Hence, ||7 || ^ \\Y\\ ^ ||X||, 
with equality only if Xq — X. 
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3. Generalized Inverses and Solutions of Linear Systems 

In what follows we will be interested in and make use of matrices BeC\(A) such that either 
AB or BA is Hermitian. The next lemma gives conditions which such C\ -inverses must satisfy and 
in fact a method of constructiong them. 

Lemma 3. Let A be an arbitrary, fixed matrix. Then every solution, B, of 



A*AB = A* 



(3) 



is in Ci(A) and is such that AB is a uniquely determined Hermitian projection. Conversely, every 
BeCi(A) such that AB is Hermitian satisfies (3). Similarly every solution, B, of 



AA*B*=A 



(4) 



is in Ci(A) and such that BA is Hermitian. Conversely every BeCi(A) such that BA is Hermitian 
satisfies (4). 

PROOF. Define E = AB. Then (3) reads A*E = A* and left multiplication by B* gives E*E = E, 
which shows E = E* = E 2 . But then (3) can be written A*B*A* = A* which implies that BeCi(A). 
Since E = E* and it is known that p(A) = p(E) , we have R(E) =R(A) and E is completely deter- 
mined by A. Conversely, if ABA = A and AB is Hermitian we have B*A*A = A which gives (3). 
The statements regarding the solutions of (4) are proved in the same way. 

In the proof of his Theorem 1, Penrose [11] includes the second part of Lemma 3. Bose [2] 
has previously shown, for the real case, that if B is any solution of (3) then AB is a unique Hermitian 
projection. The proof is quite different (cf. Rohde [14]) and neither Bose nor Rohde drew the 
conclusion that BcCi (/4). As before ([9]), we call a matrix/? a strict C\ -inverse of A \fBeC\(A) and 
B4Ci(A), or equivantly ([7, 15]), if BeC\( A) and p(B) >p(A). There exist strict C\ -inverses of 
A which satisfy (3) or (4) or both. For, if we write 



A = Q 



I) 




R 



(5) 



with Q and R unitary, where D is square, nonsingular and p(A) = p(D) , then [9] any He(Ji{A) 
must have the form 



B = R* 



D 1 U 
V W 



Q* 



(6) 



where the entries of U, V, and W are arbitrary. It is readily verified that AB is Hermitian if and 
only if U = 0; that BA is Hermitian if and only if V= 0; and that when either or both of these con- 
ditions are satisfied, p{B) > p(A) if W ¥* 0. Thus there exist solutions of (3) which are not in 
Cn(A) , solutions of (4) not in Qv(A) and solutions of both (3) and (4) other than the C 4 -inverse of A. 

The generalized inverses required by the theorem of Decell and Odell [3] are precisely those 
matrices satisfying (3), i.e., those matrices of the form (6) with U = 0. 

Penrose [11] has shown that KXM = P is consistent if and only if P=KLPNM, where L is 
any matrix in C\{K) and A^ is any matrix in C\(M). In that case, X () = K + PM+ is the solution of 
minimum norm [12]. The following theorem shows that, in the consistent case, the solution of 
minimum norm can be obtained by the use of generalized inverses in a weaker class than the 
Converse. 

Theorem 1. Let K, M, and P be given matrices such that KXM= P is consistent. Then the 
solution of minimum norm is Xo= LPN, where L is any matrix in Ci(K) such that LK is Hermitian 
and N is any matrix in Ci(M) such that MN is Hermitian. 
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PROOF. Let X be any solution. Then X {) = LPN = LKXMN is also a solution. Since LK and MN 
are Hermitian projections, we have by Lemma 2, that ||Xq|| ^ ||^l||, with equality if and only if X = Xo. 

Ben-Israel and Wersan [1] have shown that the solution of (3) with minimum norm is the 
d-inverse of A. We show how this follows from Theorem 1. 

COROLLARY I. If A is a given matrix, the solution o/*A*AX= A* of minimum norm is the C 4 - 
inverse of A. 

PROOF. Let KeC { (A*A) and KA*A be Hermitian. Then, by Theorem 1, X {) = KA* is the solu- 
tion of minimum norm. It is known [9] that XoeC%(A). Since X&A — KA *A is Hermitian, we have 
X eC 4 (A). 

In view of Lemma 3, Corollary 1 shows that among all matrices BeC\{A) such that AB is Her- 
mitian, the CVinverse of A has minimum norm. In fact among all matrices in C\ {A) , the C 4 -inverse 
of A has minimum norm. A proof of this has been indicated by Kalman [10]. We give this result 
below as a corollary of Theorem 1. 

COROLLARY 2. If A is a given matrix, the solution of AXA= A of minimum norm is the C 4 - 
inverse of A. 

PROOF. Clearly AXA=A is consistent. By Theorem 1, the solution of minimum norm is 
X() — B\ABi where B\ and B> are in C\{A) and are such that B\A and AB 2 are Hermitian. By a 
known theorem [7], XoeC 2 (A) , and plainly AX {) = ABy and X^A = B\A are Hermitian. Hence 
XoeCAA). 

Remark. One construction of the C 4 -inverse of A given by Penrose [11] is in fact B — B\AB>, 
where B\ and B> are any solutions of (4) and (3), respectively. This is a special instance of the 
theorem [7] that B = B\AB 2 is in C>(A) when B\ and B> are arbitrary elements of C\(A). In par- 
ticular, if AB 2 is Hermitian, then BeC :i (A) and if B X A is Hermitian, then BeC, v {A) ( [cf. 9]). We 
could also prove Corollary 2 by observing that among all matrices of the form (6), 



B = R* 



D-t 




has minimum norm and that, according to the discussion after (6), this matrix is A+. 

The next theorem gives a subset of C\(A ) such that elements of this subset select the solution of 
minimum norm when (1) is consistent. It also gives a subset of C\{A) which selects a least-squares 
solution of (1) and shows that the C 4 -inverse of A is the only matrix which enjoys both properties. 

THEOREM 2. Consider the system 

Ax=y (s) 

where A and y are a given matrix and vector respectively. 

(i) Let BeC\(A) and BA be Hermitian. If (s) is consistent, then xo = By is the solution of mini- 
mum norm. Conversely, if Xo = By is the solution of minimum norm for every y such that (s) is 
consistent, then BeC\{A) and BA is Hermitian. 

(ii) Let BeC\(A) and AB be Hermitian. Then x — By is a least-squares solution. Conversely, 
if x — By is a least-squares solution for every y, then BeC\(A) and AB is Hermitian. 

(iii) Let B be the C 4 -inverse of A. Thenx<) = Z?y is the least-squares solution of minimum norm. 
Conversely, if Xo= By is the least-squares solution of minimum norm for every y, then B is the 
C 4 -inverse of A. 

PROOF, (i) It follows from Theorem 1 that Xo = By is the solution of minimum norm if BeC\{A) 
and BA is Hermitian. Conversely, suppose xo = By to be the solution of minimum norm for each 
yeR(A). Then Axo = ABy=y, for all yeR{A) implies ABAz = Az, for arbitrary z, and so BeCi(A). 
Thus far we have only used the fact that By is a solution. For any z,\ety = Az, then we have By = BAz 
and ||By|| = ||B/4z||. If By is the solution of minimum norm, ||Z?y|| = ||&4z|| ^ ||z||, and that BA is Her- 
mitian follows from Lemma 1. 
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(ii) The least-squares solutions of (s) are the solutions of A*Ax = A*y. We show that x — By 
with B as in (ii) satisfies this equation. We have A*Ax = A*ABy = A*B*A*y= A*y, since AB = B*A* 
and ABA = A. Conversely, if A*ABy = A*y for every y, we have A*AB = A* and, from Lemma 3, 
BeC\{A) and AB is Hermitian. 

(iii) First, let B = A + . Then by (ii), Xq is a least-squares solution. Now let x be any least-squares 
solution. Then A*Ax = A*y from which BB*A*Ax= BB*A*y. But B*A* = AB and we have BAx 
= By=xo. Since BA is Hermitian, \\xo\\ ^ \\x\\ follows from Lemma 1. Conversely, suppose that for 
every y, Xo = By is the minimum norm solution of A*Ax = A*y. By Theorem 1 we then have Xo 
= By=KA*y for all y, where KeC\{A*A) and KA*A is Hermitian. Thus B = KA*, which as in the 
proof of Corollary 1 is the C 4 -inverse of A. 

Remark. In the proof of (i) we have essentially proved that if BeC\(A), then x = By is a solu- 
tion of (s) whenever yeR (A) and that, conversely, if x = By is a solution of (s) for every yeR (A) then 
BeCi(A). This was proved by Bose [2] (cf. [14]) and in fact Rao [13] takes this as a definition 
of Ci(A). 

The content of (iii) of Theorem 2 is known. For, Penrose [12] shows that X = A + C is the least- 
squares solution of minimum norm of AX—C. By observing that /^ + is the least-squares solution 
of minimum norm of AX = I he shows that A + is the only matrix with the required properties. We 
prove (iii) here to show how the result stems directly from the Hermitian character of A + A and 
AA + and to show the relation of (iii) to (i) and (ii). 

It is clear that any BeCv(A) meets the conditions of (i) of Theorem 2. It is precisely that 
property of the CV-inverse which was exploited by Goldman and Zelen [4J. Their proof however 
is essentially different in character. 2 With Theorem 2 in hand, we can give a rather brief version 
of their proof of the Gauss theorem (Theorem 1 of [4]) utilizing a weaker class of generalized inverse 
than the 6V-inverse. The term estimable is used here as it is used in [4|. 

THEOREM 3. Let y be a vector of random variables such that the variance-covariance matrix 
is cr 2 / and the expected value of y is /f £. The minimum variance unbiased linear estimate 6 of any 
estimable function d = q*t ) is given by 0= q*By= q*x, where B is any matrix in C\(A) such that 
AB is Hermitian and x is any solution of (2). 

PROOF. A linear estimate 6 = p*y of is unbiased if and only if the expected value E(0) of is 
E{6) = p*At i = d for all £, i.e., A*p = q. Thus is estimable if and only if qeR(A*), and since the 
variance of is (p*p)cr 2 , 6 gives the minimum variance unbiased estimate if and only if p is the 
minimum norm solution of the consistent system A*p = q. According to (i) of Theorem 2, this is 
equivalent to p = B*q where B*eC x {A*) and B*A* is Hermitian, i.e.,BeCi(A) and AB is Hermitian. 
Thus = p*y= q*By. But for any qeR(A*) we further have q*By= q*x, where x is any solution of (2). 
For, by (ii) of Theorem 2, By is a solution of (2) and any solution of (2) can be written x = By+v 
with veN(A). 

It is of interest to note that the general solution of (2) can be written asi = KA *y + (/ — KA *A )z , 
where K is any C\ -inverse of A* A and z is arbitrary. 3 This, in a different notation, is eq (3.3) of 
Goldman and Zelen [4], except that they require that KeCz{A^A) . As noted previously in this paper, 
and shown in [9], KA*eCz(A). Thus Goldman and Zelen in fact employ both the C 3 -inverse and 
the Qv -inverse and these are included in the set of all matrices which meet the conditions of 
(ii) and (i) of Theorem 2, respectively. 
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